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This aeta-analysis investigates the effects on 
achievement of type of graphing paper eaployed in displaying student 
performance data collected over tiv*. Ongoing curriculum-based 
measurement systisms employing OatM-Based Program DevelopSMnt (DBPD) 
provide a data source of 16 controlled studies with 17 effect size«. 
A total of 3,494 subjects participated in these studies, with 81 
percent of the investigations employing handica|inP<*d subjects. The 
average weighted unbiased effect sizes for six-cycle and equal 
interval paper, respectively, were .65 and .46. Hedges' analogue to 
analysis of variance indicated this difference was not statistically 
reliable. This study provides a basis for two conclusions: (1) the 
existing data base concerning methods for displaying student 
performance data indicates that type of graphing method does not 
affect student achievement reliably; and (2) this meta-analysis 
highlights the need for components of DBPD, including graphing 
conventions, to be contrasted within the context of controlled 
experimental investigations. Comparisons of DBPD exponents within 
controlled experimental studies should provide important knowledge 
about critical elements of ongoing monitoring systems for effective 
development of individualized instructional programs, implications 
for special education practice are discussed. An appendix lists the 
reports included in the meta-analysis. (A«xthor/PN) 
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Abstract 



This meta-analysis investigated the effects on achiegement of type of graphing 
paper employed in '^ ^'^Uying student performance data collected over time. The 
data source was 16 c ^rolled studies with 17 effect sizes. The average weighted 
unbiased effect sizes for 6-cyc1e and equal interval paper, respectively, were 
•65 and .46. Hedges^s analogue to analysis of variance indicated this difference 
was not statistically reliable. Implications for special education practice are 
discussed. 



Effects of Alternative Student Performance Graphing Pxocedures on Achievement 



Individualization, in which th» pacing and method of instruction art 
varitd to match »tudent»' needs (Glaser & Nitko, 1<71), is a popular notion in 
special education (see Corri-jan, 1978; Council <or Exceptional Children Delegate 
Assembly, 1983| Mann, Suiter, & McClung, 1980), and Federal Uw <PL 94-M2) 
mandates that an individualized tducational program be dtvised <or each 
handicapped pupil. The traditional strategy ior developing such programs is an 
apti tute-treatment interaction approach, whereby norm-referenced measurfs are 
•mployed to diagnose students' ability profiles, and then educational programs 
are matched to those profiles and prescribed as treatments. Uhil* such a 
diagnostic-prescriptive approach if prevalent even today <Carbo, 1983), evidence 
accumulated over the past 15 years indicates that it fails to contribute to 
differentially effective learning rates <Arter & Jenkins. 1977, 1979; Hammill & 
Larsen, 1974; Harnmi.l & Weiderholt, 1973). 

As an alternative to this diagnostic-prescriptive strategy for generating 
individualized instructional programs, ongoing curriculum-based measurement 
systems have been developed (Deno & Mirkin, 1977; LindJiey, 1977; Lovitt, 1977; 
Mirkin et al., 1981; Uhit & Haring, 1980). U.th these systems, hereafter 
referred to as Data-Based Program Development <DBPD), initial educational plans 
•re vifwed as hypotheses concerning effective treatment. Therefore, during 
treatment implementation, data are collected in an ongoi ng manner , graphed, and 
analyzed to evaluate hypotheses and revise and improve programs. This leads to 
tmpiriolly derived and validated individualized educational programs <Deno & 
Mirkin, 1977). 

Evidence indicates that the use of DBPO improves student learning. Fuchs 
and Fuchs <in press) reported .4n average effect ujt of .70 ior students whose 
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programs were developed systematically and empirically over time. This indicates 
that, in terms oi the standard normal curve and an achievement test scale with a 
population mean o^ iQO and a standard deviation o^ 15| the use o^f DBPD can be 
expected to raise the typical achievement outcome score from 100 to 110.50, or 
from the 50th to 76th percentile. 

One DBPD component associated with student achievement is graphing. When 
student data are charted rather than simply recorded, achievement improves 
approximately .50 oi a standard deviaticri (Fuchs & Fuchs, in press>. in fact, 
wivhin DBPD specifically and applied behavior analysis generally, agreement 
prevails that graphing is critical: It assists in organizing data for formative 
evaluation, provides a detailed numerical summary and visual description of 
performance, and facilitates communication of program results <Tawney & Gast, 
19S»i). 

Despite concurrence on the importance of graphing, salient differences 
exist concerning specific graphing conventions, including the type of paper 
employed. Some programs advocate the use of ratio or logarithmically scaled 
graph paper <e.g., Lindsley, 1977; Uhite & Haring, 1980), where the rate scale is 
adjusted to display proportional changes in student behavior. For example, Ihe 
change from 10 to 20 is identical in distance to the change from 20 to 40 or from 
40 to 80. In contrast, developers of other DBPD systems support the use of 
equal interval, or conventional, graph paper <e.g., Deno&Mirkin, 1977). 

In selecting between alternative DBPD procedures, such as graphing paper, 
three basic considerations are technical properties, logistical features^ and 
effects on student achiei^ement <Deno, Mirkin, & Fuchs, 1982). Available research 
and competing arguments for each consideration relevant to the two types of 
graphing paper are presented below. 
Technical Properties 

Proponents of logarithmically scaled paper contend that an important 
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Justification for this graph paper Is technical; that is, the ratio scale more 
accurately reflects the proportional way in which natural change occurs than does 
equal interval paper (Uhite & Haring, 1980>. Yet, findings of the only 
identified, relevant, empirical contrast of the two graphing methods fail to 
support this contention. Specifically, Marstor (1982) compared the prediction 
capabilities of logarithmic and conventional charts. He col lected weekly 
performance data in reading, spelling, and written expression for 10 weeks. 
Theni he charted student performance data over weeks 1 through 7 on equal 
interval and on ratio scaled paper. Next, based on the trends indicated on each 
graph, he calculated predictions concerning performance during weeks 8 through 
10. Finally, he compared each prediction to actu2l student performance for weeks 
8 though 10 and found that, for each academic area, predictions were 
significantly better when based on the equal interval, rather than the ratio 
scaled, graphs. This finding tentatively suggests that change in these academic 
areas occurs additively rather than proportionately, questioning the basic 
technical rationale for the use of logarithmic paper. 
|,OQist ica] Features 

Regarding logistics, two basic arguments exist, one supporting each type 
of graphing method. Proponents of the logarithmic approach purport that the 
corresponding paper is logistically superior because a single c.^rt can be used 
display all relevant behaviors, given the large behavior range covered on one 
graph, and relatedly, that a standard chart can facilitate comparisons among 
different behaviors (Uhite & Haring, 1980). On the other hand, some contend that 
equal interval graphs facilitate data analysis (Tawney & Gast , 1984), basically 
are easier for students and teachers to understand, and that this understanding 
may lead to more consistent implementation of DBPD (Mirkin, Fuchs, it Deno, 1982). 
Despite this controversy surrounding the relative logistics associated with the 
typ^s of graphing paper, we know cf no empirical conxrast of teacher and student 
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concerns. Therefore^ there if no objective basis to support the relative 
logistical merits of either graphing approach. 
Effects on Achievement 

Despite continuing disagreement concerning which type of graph method is 
technically and logistiallx superior, there has been relatively little attention 
directed toward which type of graphing method leads to improved student 
achievement. In a search for relevant findings, only one related report was 
identified. Brandstetter and Merz (1978) conducted a series of two studies. The 
first compared gains made while charting scores on linear graphs with g^ins made 
while simply recording raw scores. The second compared gains associated with 
charting scores on ratio scaled graphs with those related to the simple recording 
of raw scores. Unfortunately, no attempt was made to compare the effectiveness 
of graphing on linear and ratio scaled graphs. Furthermore, th<e children 
employed in the two studies were neither randomly assigned nor similar to each 
other, mal<ing it impossible to draw valid comparisons between the educational 
effects of the two types of graphs. 

Therefore, the currently available data base for selecting the superiur 
graphing method is inadequate. Only one study addresses the prediction 
capabilities of the methods, with little available information concerning other 
technical properties, and ther« is no data base on the relative logistical 
strengths and weaknesses of the methods. Moreover, the data oase concerning the 
most important criterion for selecting a type of graph i ng method, effect on 
student achievenent, is scant. Nevertheless, a diverse literature on educational 
effects of DBPD programsj, in general, is available. The methodology of 
meta-analysis allows for the comparison of effects across different studies along 
the factor graphing method. Consequently, the pu'^pose of the current 
investigation wcis to conduct a meta-analysis of the effect of graphing method on 
academic achievement. Such an analysis should clarify whether these two well 
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known approaches contribufe differentially to student achievement. 



Method 



Search Procedure 

The search for pertinent studies to include in the f;.eta-anal ysi s 
comprised four steps. First, employing the Thesaurus of Psychological Index 
Terms <APA, 1982), multiple descriptors were generated for key topic-related 
terms. For example, student achievement alternately was represented by "student 
progress" and "educational effects." Second, these terms facilitated a ccnputer 
search of three on-line data bases: <a) ERIC, a data base of educational 
materials from the Educational Resources Information Center consisting of 
abstracts from Research in Education and Current Index to Journals in Education; 
<b) Comprehensive Dissertation Abstracts; and <c) Psycholooical Abstr^cts> 
Third, employing similar ke> descriptors, a manual search was conducted of five 
educational Journals for the years 1973 to present. These journals were*; 
American Educational Research Journal. Journal of Learnino Pi sabi 1 i t ies t 
Journal of Precision Teachino. Journal of Special Educationt and Learning 
Disabi 1 i ty Quarterly. Fourth, titles in the reference sections of investigations 
discovered by these efforts were explored fo*" additional studies. 



Criteria for Relevant Studies 

A study was considered for inclusion if it employed a control group to 
evaluate the effects of^BPD on the academic performance of elementary and/or 
secondary students. DBPD was defined as curriculum-based data collection that 
occurred at least twice weekly, with decisions concerning the adequacy of 
programs formulated on an individual, not group, basis. Studies were excluded 
that <a} monitored nonacademic behaviors, <b) primarily focused on the use of 
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behavior modif icationi while employing time series to test e^^perimental effects, 
<c) provided test feedback only to students, and/or <d) employed preschool or 
college-age subjects. 

The search yielded 28 studies that met the criteria established for 
inclusion. From these studies, 12 were eliminated because of insufficient ceta 
for calculating effect sizes, 'eaving 16 studies that were employed in the 
analyses described b»1ow. 

Data Extracted from Each Study 

Guidelines were established to ensure that each relevant effect size was 
counted only once in analyses, and that different papers reporting results of the 
same study were grouped within analyses as one investigation.^ 

Results of the studies were transformed to a common metric, effect size, 
defined here as the difference between the treatment means, divided by the 
cont.ol group standard deviation. For purpose of analysis, an effect was gis n 
a positive sign if subjects achieved greater scores in the DBPD treatment. For 
studies reporting relevant means and standard deviations for both groups, effects 
sizes were calcuUted from these measurements. For studies not reporting means 
and standard deviations, effect sizes wet e calculated from other statistics such 
as F or £ values <see Glass, McGaw, & Smith, 1981). When pretest differences or 
analyses of covariance were reported, alternative procedures for calculating 
effect size were used, as possible, to control for initial student differences. 

Each effect size was converted to an unbiased effect size <UES) to 
correct for inconsistency in estimating true from observed effect sizes <Hedges, 
1981). The difference between the observed and UESs was neglible <X = .019, SD = 
.025) as has been demonstrated elsewhere <Bangert-Drowns, Kulik, & Kulik, 1983). 
Nevertheless, UESs were employed to insure the mathematical tractability of the 
data. 
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Da ta aQQreQatlon, UESs were aggregated at the study level for di-fferent 
t/pes of graphing method. Therefore, one UES per study was reported <with the 
exception of the Brandstetter and Merz [1972] article for which there was one 
effect size for equal interval and one for ratio scaled paper). In aggregating 
these UESs for a measure of central tendency for each tyoe of graphing method, 
weighted averages were calculated to account for the variance of the UESs <see 
Hedges, 1984). 

To describe study features pertinent to the current investigation, the 
type of graphing rr«thod employed in each study was identified and coded. This 
variable had two levels, equal interval paper and ratio scaled paper. Two coders 
independently coded 10 of the 16 studies <637.). Percentage of agreement* for the 
coders was 100. A previous investigation <Fuchs & Fuchs, in press) explored 
methodological quality of the studies and identifed no relation between effect 
size magnitude and study quality. 

Characteristics of the Sample 

Of the 18 references in the Appendix, which represent 16 separate 
investigations, there are 4 dissertations, 9 unpublished studies, and 5 Journal 
articles. Among the published papers, 2 appeared in Exceptional Children. 2 in 
American Educational Research Journal, and 1 in American Journal of Mental 
Deficiency. A total of 3494 subjects particiated in these studies, with 81X of 
the investigations employing handicapped subjects. Of these handicapped pupils, 
91'/. were mildly to moderately handicapped and 97, were severely handicapped. The 
grade level of these subjects ranged from 1 through 12, with a median grade level 
of 3.85. Among the 16 investigation^ 7 <44y.) focused solely on the area of 
reading, 3 <197.) on reading and math, 2 <137.) on math alone, and 1 <67) each on 
(a) spelling, <b) high school content areas, <c) reading and spelling, and <d) 
reading; math, and spelling. 
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Rgsul ts 

Sixteen studies with 17 assoc inted mean effect siz€s were identified for 
the meta-analysis. These studies with associated average effect sizes are 
presented in Table 1. Of these UESs, 7 were related to equal interval graph 
paper and 10 were associated with ratio scaled paper. 

To examine the relation between UES and type of gr aphing method^ Hedges's 
<1984) chi-square analogue to analysis of variance was employed. Uhen 
conventional analysis of variance is conducted on effect sizes^ problems exist 
because of the possibility that systematic variance will be pooled into the 
estimate of error variance. Moreover^ violation of the homoscedast ic i ty 
assumption is severe in research synthesis^ and there is little reason to believe 
that the usual robustness of the F test will prevail <see HedgeCi 1984)* Thusi 
Hedges^'s analogue to analysis of variance was employed to avoid these conceptual 
and statistical problems. 

Results revealed no significant effect, X^<1| N = 17) = 3.6, ns , 
indicating no reliable difference in achievement between the different graphing 
methods. For equal interval paper, the mean weighted UES was .46, with a 
variance of .0072. This mean UES was, itself, statistically significantly 
di f ferent from zero, z = 4.79, £ < .001 . For rat i o seal ed paper , the mean 
weighted UES was .63, with a variance of .0016. This mean UES also was 
statistically significantly different from zero, z = 16.25, £< .001. 



Insert Table 1 about here 
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Discucsio r 

DBPD represents an effective alternative strategy for developing 
individualized instructional programs for special education students <Fuchs & 
FuchSi in press). Moreover i o^^aphing student performance data is a critical 
component of effective DBPD (Fuchs & Fuchs, in press). Nevertheless, previous 
research fails to provide in adequate data base for determining whether a type of 
graphing method is superior for displaying and evaluating data within DBPD. 
Therefore, the purpose of this study was to integrate quantitatively the 
available research on DBPD to assess .he effect of type of graph paper on student 
achievement • 

Results indicated that type of graphing nethod employed to display 
student performance data did not produce a statistically significant effect on 
student achievement. Additionally, the difference between the mean effect sizes 
dssoc i^ted wi th the graphing methods is of little practical importance (Cohen, 
1977); The weighted mean effect size for ratio scaled paper was appro)^ imately .2 
of a standard deviation higher than the weighted average effect size associated 
with equal interval paoer. This indicates that, in terms of the standard normal 
curve and an achievement test scale u^ith a population mean of !00 and standard 
deviation O'^ !5, one might expect the use of ratio scaled gaph paper to raise the 
typical achievement outcome a relatively small degree, from 100 to 103. 

Therefore, this study provides a basis for two conclusions. First, the 
existing data base concerning methods for displaying student performance data 
Indicates that type of graphing method does not affect student achievement 
reliably. Consequently^ as practitioners design systematic formative evalation 
procedures wi th which to formulate individualized educational programs, they may 
employ personal preferences and logistical considerations. Second, this 
meta-analysis highlights the need for components of DBPD, including graphing 
conventions, to be contrasted mm th i n the content of controlled experimental 
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invest >gAtionss Whereas meta-analysis serves to integrate and quantify effects 
across studies of different methodology and conceptual izatiOHf it is possible for 
systematic variations to occur between groups of studies constituting important 
contrasts. Thereforei comparisons of DBPD components within controlled 
experimental studies should provide important knowledge about critical elements 
of ongoing monitoring systems for effective development of individualized 
instruct ional programs. 
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Footnotes 



^One paper authored by Haring (1971) and two additional reports by Haring 
tc Krug (1975a9 1975b) described aspects o^ the same investigation. Thereforei 
although it is reported that 1^ studies were employed in the meta-analysiSi 18 
appear in the Appendix due to the separate listing O'f the Haring and the Haring 
and Krug papers. 

2 

Percentage O'f agreement was calculated using the "following ^formula 
(Coulter cited in Thompson, Uhite, & Morgan » 1982) s Percentage O'f agreement » 
agreements between ooserver A & observer B/(agreements between A B ^ 
disagreements between A & B omissions by A ^ omissions by B >. 
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Table 1 

Study Citations and Weighted UESs by Graphing Convention 



Graphino Convention/Citation Uleiohtfd UES 

Equal Interval Paper 

King, Deno, Mirkin, & Uesson (1983) - .37 

Tindal, Fuchs, Christenson, Mirkin, & Deno <1981) - .09 

Skiba, Uesson, & Deno (1982) .21 

Mirkin (1978) .43 

Mirkin, Deno, Tindal, & Ku^hnle (1980) ,B6 

Fuchs, Deno, & Mirkin (1984) 1.00 

Brandste»ter & Merz (1978) 1.06 

Six-Cycle Paser 

Brandstetter & Merz (1978) .13 

Frumess (1973) .21 

Beck (1981b) .33 

Beck (1979) .45 

Beck (1976) .48 

Beck (1981a) .50 

Haring (1971)/Haring & Krug (1975a, 1975b) 1.04 

Dubrule (1984) 1 .18 

Bohannon (1975) I .58 

Bruening (19^b) 1 .99 

0 
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